This invention relates to manipulation of visual images from large and detailed datasets. The need for this invention grew out of projects related to creation and manipulation of visible human datasets of developed from visible human research (VHR) projects that have been ongoing in many countries for over a decade. The invention has particular application to computer assisted medicine and medical education, but its application is not limited to medical applications.
The present invention relates specifically to volume rendering of data. More particularly, the present invention relates to 3D-textured axis-aligned slices in volume rendering.
Volume rendering is a technique for directly rendering a field of 3D scalar samples without first fitting geometric primitives to the samples. Common methods of volume rendering include ray-casting, volume splatting, and 3D-texture mapping.
Geometric primitives refer to polygonal representation of points, lines or surfaces. In particular, polygonal representation of surfaces is generally done by using triangle-based meshes or NURBS. Ray-casting is a technique for direct volume rendering by emitting a ray from a viewpoint to every screen pixel and then accumulating sample values along the ray.
Volume splatting is a technique for texturing a terrain using high resolution localized tiling elliptical Gaussian disks which transits nonlinearly. Splatting makes heavy use of a computer graphics processor which is expected to render each triangle many times to combine textures in a frame buffer. This requires heavy use of fill-rate and triangle-rate.
Referring to FIGS. 1A and 1B, there are the two different implementations of 3D-texture mapping, or 3D texturing through blending, which are other direct volume rendering techniques that utilize 3D texture memory on a graphics board: view-plane-aligned stacks of slices using 3D texturing (FIG. 1A) and axis-aligned stacks of slices using 3D texturing (FIG. 1B). Volume rendering techniques are of particular interest.
The overall work-flow of 3D texturing is as follows: Static signals are obtained from the original dataset; after having performed various texture threshold operations, volume picture elements or voxels of the volume data are initialized; then texture-mapped geometric primitives are rasterized into blocks of fragments based on the viewable region. Each such block has information on vertex and texture. Finally, blocks so obtained are handled by a conventional graphics pipeline.
The processing can be divided into three phases: vertex processing, rasterization and fragment processing.
1) Vertex Processing: The object of vertex processing is to process the point constructed from the volume dataset in the whole rendering process. The processing includes resizing, rotation, translation, spatial positioning or texture coordination of output points, and generating a color signal. In static volume rendering of a 512-by-512-by-512-point volume dataset, it takes four points for each slice, and hence it takes 2048 points in total.
2) Rasterization: The object of rasterization is to transform geometric primitives into fragments expressible in a matrix including and above a main diagonal, as shown in FIG. 2. Each fragment is associated with depth, color and texture coordinate. Having initialized the volume dataset, the volume is rendered by a cumulative blending of a stack of slices. During rasterization, fragments are generated. For instance, if the volume is rendered orthographically, a 512-by-512-by-512-point volume dataset do generates 512 times 512 times 512 equals 134,217,728 fragments which demand a huge amount of rendering resource.
3) Fragment Processing: The object of fragment processing is to assign texture color to every fragment of the stack of slices. Since the texture is constructed from discrete units, extra processing, e.g. interpolation, is required to map a texture value onto each fragment. In cases when the edge of texture units is mapped onto the edge of a fragment, extra texture processing is required. Due to these calculations, texture mapping can be very computational intensive.
The basic idea of standard static 3D texturing is to render image onto a two-dimensional display by accumulating voxel values from a stack of texture mapped slices. Its advantage is simple in implementation and can be easily accommodated with current graphics pipeline techniques. Taking the known OpenGL 3D texturing kernel as an example, one need only initialize the volume texture from the dataset and initialize every vertex of the stack of slices. The rest of the work can be left to OpenGL's rendering pipeline.
FIG. 3 outlines the work-flow of prior art static volume rendering in a volume rendering system 10. It involves separating volume data 100 into volume vertex data 102 and volume texture data 104, processing the volume vertex data in a vertex processor 106 to obtain a vertex transformation 107, then processing the vertex transformation in a primitive assembly processor 108 to obtain a primitive assembly 109, applying a rasterization and interpolation processor 110 to obtain a rasterization 111, and processing the result with the volume texture data 104 in a fragment processor 112 where fragments are then assembled in a frame buffer 116 to yield a map texture output 118. Disadvantageously, this method requires the unused signals derived from the volume picture elements to always be passed along the rendering pipeline. Signals refer to a set of RGBA color entries. This leads to an inefficient use of hardware resources.
To maintain the completeness of the dataset, static signals are retained. In the data preprocessing phase, for datasets in common volume visualization applications, it has been determined that at least ten percentage of the static signals are unused, i.e., not used for final rendering. In addition, threshold control or other texture operations may generate new but non-contributing static signals. These unused signals are usually rendered with zero alpha value, in other words, fully transparent. Thus, keeping these unused signals can greatly degrade rendering performance.
In a real-time rendering phase, for a specific positional and orientational status of a dataset, occlusion due to preceding slices also generates unused signals. One of the common remedies is to reduce the number of slices rendered when the volume data is translated, rotated and rescaled. This enables the volume set to be moved or re-oriented smoothly, but at much lower visual quality.
One of the main factors lowering the volume rendering performance is the processing of huge amount of fragments during the rasterization phase. Since complicated computations in texture mapping must be performed on every fragment, the number of fragments that are processed can readily exhaust the texture memory. Thus, rendering very large datasets using static volume rendering has been infeasible on a conventional consumer-grade personal computer.
What is needed is a method and a system for rendering very large datasets of graphics data that is better suited to use of a conventional consumer-grade personal computer.